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Abstract 

This is an open letter on the paper "Adaptivity and computational 
complexity in the numerical solution of ODEs" by Silvana Ilie, Gustaf 
Soderlind and Robert M. Corless. We sent this letter to the authors 
in August 2008. 

Dear Silvana, Gustaf and Robert, 

Your paper "Adaptivity and computational complexity in the numerical 
solution of ODEs" recently appeared in the Journal of Complexity. After 
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studying this paper, we found a few points that we would hke to discuss with 
you. 

In your paper you study the numerical solution of iVPs and BVPs for odes. 
In the abstract you claim that (under some mild conditions) "an adaptive 
method is always more efficient than a non-adaptive method" . 

You mention "computational complexity" in the title and one of your 
keywords is "information-based complexity" . Moreover, you cite the book of 
Werschulz [9]. It seems therefore that your paper is meant as a contribution 
to complexity theory and, more particular, to IBC. 

The first two sentences of the introduction read as follows: 

The complexity of numerical algorithms is central to the assess- 
ment of computational performance. For some algorithms, like in 
linear algebra, the complexity is well known and established; for 
others, like ordinary differential equations (odes), the complexity 
is still open to analysis. 

We agree with the first sentence. However, you are confiating two different 
concepts: the cost of an algorithm and the complexity of a problem. The 
complexity of a problem is the cost of an optimal algorithm. In other words, 
the complexity is the minimal computational cost of solving the problem, said 
minimum being taken over all algorithms that solve the problem. Complexity 
is not a property of any particular algorithm; it is a property of the problem 
under consideration. 

More important is the following. We do not currently know the complex- 
ity of many problems of linear algebra. Probably the most basic problem from 
linear algebra is matrix multiplication, which is equivalent to solving linear 
equations. Unfortunately, we do not know the complexity of this problem. 
We know only bounds on the complexity. This famous problem has its own 
history, which has been enriched by the contributions of Strassen, Pan, Cop- 
persmith and Winograd. They would be quite surprised to see that someone 
claims that the complexity of this problem is well known. 

On the other hand, if we interpret complexity as being equivalent to 
cost, then we agree that the cost of some algorithms in linear algebra is 
known. But we also know the cost of many other algorithms for ODEs, 
such as Euler, Runge-Kutta, and multistep methods. This suggests that the 
notion of complexity used by you is different than the notion of cost, as it 
should be. 
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Of course we believe that the complexity of odes "is still open to anal- 
ysis" since we do not know everything concerning the complexity of ODEs. 
Nevertheless, it is quite surprising for us that you almost completely ignore 
what we do know about the complexity of solving odes. Kacewicz proved 
many complexity results for ODEs, mainly for iVPs. We recommend his pa- 
per [1] on the complexity of iVPs for ODEs and stress that most of the upper 
bounds are proved using adaptive algorithms. Moreover, Kacewicz wrote 
another paper [5], in which he compares the power of adaptive and non- 
adaptive information and proves that adaption is exponentially better for 
d-variate odes. 

There is no IBC result saying that adaption never helps, see also [HI E] ■ 
Werschulz proved many complexity results for odes, mainly for EVPs. Other 
authors proved many more results. None of these results can be found in 
your article, although the book of Werschulz [9] is cited. 

You probably heard the result that adaption does not help but this result, 
as all mathematical results, only holds under specific hjrpotheses. In this case, 
we must assume that 

• we want to approximate a linear operator 

• defined on a convex, balanced class, and that 

• we use linear functionals for approximating the linear operator. 

Then adaption can be only better by a factor of at most 2. Note that the 
operator associated with odes is non-linear, since the solution of ode's of 
the form u'{t) = f{u{t),t) depends non-linearly on /. So, the theorem on 
adaption cannot be applied for odes. In fact, the results in Kacewicz |5] 
dramatic prove that adaption can be significantly better than non-adaption. 
Your introduction continues: 

In the former case, the problems are "computable", meaning that 
(theoretically) the exact solution can be obtained after a finite 
number of operations, and this operation count then becomes a 
measure of the complexity. By contrast, for problems in analysis 
we can only compute approximate solutions converging to the 
exact solution. This makes an assessment of complexity more 
difficult, as the algorithmic complexity will depend on problem 
characteristics as well as the requested accuracy. 
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Of course, the necessity of computing approximations, rather than exact 
solutions, is not restricted to problems (such as integration) from analysis. 
There are also problems from linear algebra (such as eigenproblems) where we 
only can compute an approximation of the solution, despite the fact that we 
have complete information. Hence it doesn't matter whether you're talking 
about analysis or linear algebra; there are times when we need to compute an 
approximation whose error (in some norm) is bounded by some positive e. 
This e depends on the application, and might be very small or relatively 
large. 

Your introduction continues further: 

In differential equations, adaptive algorithms are of fundamental 
importance. Such algorithms attempt to minimize some (usu- 
ally coarse) measure of complexity, subject to a prescribed accu- 
racy criterion and the problem properties encountered during the 
computation. This is generally done by using non-uniform dis- 
cretization grids in order to put the discretization points where 
they matter most to accuracy, while keeping their total number 
small. 

Naturally, for some problems, uniform grids might be optimal 
from the point of view of complexity, e.g., if one considers FFT- 
based algorithms for Poisson's equation on a rectangular domain. 
For linear problems, similar considerations led Werschulz to ques- 
tion whether adaptive methods are more efficient, using a topo- 
logical argument to show that the efficiency gain would be limited 
to a factor of two O pp. 38-39]. In this paper, however, we will 
prove that adaptivity is better than non- adapt ivity. 

You never explain what you mean by "some measure of complexity". 
Moreover, you also do not define your notion of adaptivity. One has to 
read between the lines to figure out what this really means; it seems to us 
that you use the word adaption to characterize methods that are based on a 
non-uniform grid. 

First of all, we admit that any of us are free to use the term "adaption" 
to mean whatever we want. However, if someone wants to compare her/his 
results with the IBC literature, it seems necessary to use the same definition 
of adaption as is used in IBC, or at least to point out differences. Since your 
paper suggests that the IBC result about adaption is wrong, it is a pity that 
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your paper does not precisely formulate this result. Note that this result has 
a venerable history. Back in 1971, Bakhvalov [1] proved a version of this 
result for the approximation of linear functionals. The result was proved in 
full generality for linear operators independently by Gal and Micchelli [3] 
and Traub and Wozniakowski [8]. This last result is the one mentioned by 
Werschulz P pp. 38-39]. 

Since your paper claims that the IBC result on adaption is wrong, you 
should indicate an error in the proof or a counterexample. At the very 
minimum, you should state this wrong result precisely. Not surprisingly, 
we think that the IBC result on adaption is perfectly fine. It would be very 
surprising if someone would find an error in such an old and elementary result. 
After all it has been checked by many people. We are also surprised that 
even today, after 27 years, the precise statement of this result on adaption is 
not better known outside the IBC community. 

The notion of adaption used in IBC has nothing to do with uniform or 
non-uniform grids. Information is non-adaptive if we use, say, function values 
at sample points that are the same for all functions from a given class. They 
can be from uniform or non- uniform grid or from any set. Information is 
adaptive if sample points vary with functions from a given set. So Gauss 
quadrature uses non-adaptive information for univariate integration for a 
class of smooth functions, and Newton, secant or bisection uses adaptive 
information for solving univariate non-linear equations for appropriate classes 
of functions. 

We want to indicate that your paper does not contribute to the complexity 
theory for DDEs. In the introduction, you say: 

In this paper we analyze the complexity of solving odes using 
adaptive one-step methods based on local error control. 

Here you admit that you only want to study a very specific class of al- 
gorithms. In complexity theory, however, we want to discuss all algorithms 
and, in particular, find optimal algorithms. If we restrict the class of algo- 
rithms to "one-step methods based on local error control", then we cannot 
expect to obtain sharp complexity results. 

In Section 2 you present the problem as follows: 

We shall consider the problem of solving an ode, written as an 
operator equation 2,{u) = f with either initial or boundary con- 
ditions. 
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You do not introduce a class of such problems. Such a class might be 
defined by specifying certain properties (say, smoothness) of the functions / 
for which we wish to solve the problem £^{u) = f. The complexity depends 
on the domain of a particular problem, and it may vastly change if we change 
the domain. You only discuss a single ode. There is no such a thing as a 
complexity theory for the computation of a single object. For any kind of 
complexity theory one has to describe a class of problems under considera- 
tion. In the paper you prove results, in particular about "optimal" grids. A 
typical result, such as Theorem 3, starts with 

Let p be the order of the method, . . . 

You then assume that a certain method is used and only discuss variants of 
the same method with different grid points. Again, such statements are a 
contribution to the cost analysis of a particular method, but not to complex- 
ity theory. After all, you do not even aim to present optimality results for the 
class of all algorithms. You do not prove a result that your method is better 
than a suitable method using non-adaptive grid which would be needed for 
a complexity result. 

Theorem 5 is about the "minimum number of grid points" and hence 
looks like a complexity result. Again, it is not since you restrict yourselves 
to the study of a very specific class of algorithms, which is applied to a single 
operator equation. In a way, you admit this shortcoming on p. 354 when 
you say that Theorem 5 is about the cost to solve the ODE "with the given 
method". 

Complexity theory is different. We do not want to solve a single prob- 
lem instance "with the given method". The aim of complexity theory is to 
construct and to analyze optimal algorithms for the solution of classes of 
problems. 

On p. 355, you claim: 

This supports the "conventional wisdom" and resolves the com- 
plexity controversy, |9l p. 124]. 

Unfortunately, the adaption problem is not even touched in your paper. 
Obviously the question "For which problems are adaptive algorithms supe- 
rior?" is very important. Some answers to this question, as well as exact 
definitions, can be found in the texts that we cite in this letter. 

We hope to get your comments on the points we raised in this letter. 
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With best regards, 

Erich Novak 

Henryk Wozniakowski 
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